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[Para 6] This invention is an operating method and a location-based apparatus to record, 
process reproduce a participatory and experiential dynamic environment, including three- 
dimension and holographic environment for a person to perform various activities with the 
said digitally rendered environment in real-time over wireless communications network 
including, but not limited to, worldwide interoperable microwave access network, Wireless 
Wide Area Network, Wireless Broadband, Long term Evolution or 4C or any 802. xxx over 
the air (the "wireless communication") network. 

Heading 

DESCRIPTION OF RELATED ART 

[Para 7] A location-based interactivity in a 3-D (three dimension) display is different from 
existing stereoscopic image systems, which uses extra equipment such as polarizing 
eyepiece or uses a lenticular sheet with separate methods for audio and visual inputs 
therefore create inconsistency between audio and video while interacting with the user. 
[Para 8] A location-based interactivity in a 3-D (three dimension) display is different, from 
DFD method, as discussed in US Patent 7,425,072 entitled "Method and apparatus for 
displaying 3-D image" where a series of 2-D images are composed of section-images of a 
3-D object, are successively displayed at different positions in a depth-direction. However, 
these images are fixed and not dynamic to create three-dimension environment that is 
transported from another location using wireless network. 

[Para 9] Another example is discussed in US Patent Application 2004006741 9 entitled 

' ' . • t. ' 

"Holographic recording medium". This document discloses holographic recording medium 

and, particularly, to a holographic recording medium in which data can be three- 

dimensionally recorded as a hologram. However, this method uses optical device and 

therefore does not provide real-time interactivity with the user over wireless communication 

« * 

network. 

* • , * 

Page 2 of 22 



[Para 10] Another example is discussed in the US Patent 7,471,431 entitled "Hologram 
device and hologram recording/reproducing method". This document discussed a hologram 
device records information of an interference fringe of signal light and reference light onto 
a hologram recording medium and reproduces the recorded information by emitting the 
reference light onto a recording area of the hologram recording medium. This method is 
about recording and re-producing data recorded in a hologram device. However, the 
present invention is advantageous to provide a platform for real-time audio-video 
interactivity with the three-dimension environment. 

[Para 1 1] Another example is discussed in the US Patent 7,470,028 entitled "Image 
projection device and method". This document discussed A device for forming an image on 
a screen is described that comprises a coherent illumination means, an electrically 
addressed spatial light modulaitor means located in the path of light from the coherent 
illumination means, means for producing computer gerierated hologram images for display 
on the electrically addressed spatial light moduiator means, and optics to direct light 



diffracted by the electrically addressed spatial light modulator means to the screen. 



[Para 12] In essence, the prior inventions are either recording medium, projection system or 



reproducing apparatus for three-dimension images. The methods like defracted light. 



1 » 



optical medium or pixel count are used to record process and reproduce three-dimension 

*'■ • « " ■ .... 

images that are capable of processing static images to form three-dimension images or 



hologram, however this invention has focused on dynamic images using multi-dimensional 
whole-part method to record, transport and reproduce digital images, 

. ...L : 1 . : it: I 
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Heading 

SUMMARY OF THE INVENTION 

[Para 13] It is therefore an object of the prese.nt invention to provide a location-based 
system (the "apparatus") for and a method of recording and reproducing participatory, 
experiential environment including three-dimension, photo-realistic simulation and 
holograms environment (the "environment"), integrated with audio data and sensor data, 
used for performing activities over wireless communication network. 
[Para 14] To achieve the above object, there is provided in accordance with the present 
invention an apparatus for reproducing an environment including a recording system for 
recording a plurality of light beams emitted from one light source in a computing system 
(the "recording medium"); a reproducing system for reading information recorded in the 
recording medium that reproduces light beams displayed on a plurality of display systems 
(the "screens"); a plurality of audio inputs and outputs; a medium control system for 
synchronizing the recording medium for recording, reproducing a plurality of interference 
patterns when information is recorded on and reproduced from the recording medium and 
audio input outputs; and a wireless communication network. 

[Para 1 5] In another aspect this invention is a method of providing fully integrated audio- 
video communication with response on a stimulus and displaying results as environment in 
a manner corresponding to the user's movements and engagements with the physical 
movements and position using multi-dimensional whole-part pattern. Multi-dimensional 
whole-part method allows the apparatus to store, transport and re-produce audio, video 
and sensor information from both performer of the activity (the "user") and the environment 
over a wireless communication network. This dynamic environment enhances the 
experience of the interaction between the user and the environment indicating a 
communication state. 
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Heading 

BRIEF DESCRIPTION OF THE DRAWINGS 

[Para 16] For the present invention to be clearly understood and readily practiced, the 
present invention will be described in conjunction with the following figures wherein: 

(1) FIG. 1 : The top-view of the apparatus - the location-based system. 

(2) FIG. 2: The inside view of the apparatus according to an embodiment of the 
present invention 

(3) FIG. 3: A block diagram of the physical structure of an integrated Projection 
Camera Wall 

(4) FIG 4: is the pictorial illustration of a physical environment reflected in a three- 
dimensional environment according to the method of the present invention 

(5) FIG 5: is the embodiment for recording and reproducing interactivity 

(6) FIG 6: is the block diagram data flow diagram illustrating signal processing 

(7) FIG 7: is the pictorial illustration of a user interactivity with the environment 
according to the method of the present invention 

Heading 

DETAILED DESCRIPTION OF THE INVENTION 

[Para 1 7] The present invention is, generally speaking, more suitable for deployment in 
conjunction with the method of storing, processing and creating interactive, participatory 
experiential (the "IPE") displays that allow users to observe dynamic environment. For 
reference, we have two actors: a) the person actually performing an activity (the "user") and 
therefore interacting with the digitally rendered screen In the apparatus; b) an external 
physical object, illustrated as another person who is physically not present in the apparatus. 
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[Para 1 8] A diagrammatic representation of a preferred embodiment of the apparatus in 
accordance with the invention is shown in FIG. 1 . The preferred device (101) has four major 
components: a)high-speed projector with plurality of 3D digital video camera (102) that 
move vertically and horizontally to capture dynamic visuals of objects; b) computing 
equipments (103) including but not limited to computer server, software, processor and 
memory; c) audio input (1 04), such as microphone, which is processed and relayed by the 
computing device (103); d) audio output, such as speakers, which connected, processed and 
relayed by the computing device; e) a wireless communication network (106). 
[Para 1 9] A more detail view of the apparatus (1 01 ) is depicted in FIG 2. wherein all the six 
sides have high-speed projector with 3D digital video camera (the "projector camera wall") 
fixed on the wall(102). Since the shape of the apparatus is designed, however not 
exclusively designed so, as oval-shaped chamber, the projector camera wall (1 02) covers all 
aspects of the object in that apparatus to form dynamic images. In the middle of the 
apparatus, there Is a audio-video-sensor processor (the "processor") as a unit (201) to 
synchronize sound, light and sensor information in order to project the image on the screen 
(202). The apparatus also has wireless communication network (204) to receive and 
transmit data to the processor (201) from external sources in order to project on the screen 

(202) . The processor (201) may generally consist of a series of microprocessors, memory, a 
two-dimensional sensor which obtains two-dimensionally modulated image data, and 
converters which converts analog data obtained by the light-receiving unit into digital data. 
The processor in turn communicates with a set of computers and application components 

(203) that are housed underneath (the "chamber") as one component (206) of the apparatus 

(101). There are several recording means (205) by which the real-time IPE could be achieved 

by manipulating and adjusting, focus, exposure and speed of movement of digital camera 

lenses to capture the state of the object in the apparatus (101). A light receiving controller 

(the "controller") manipulates various components such as focus, exposure, speed, 
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directions and communications to control plurality to digital video cameras. The controller 
in turn communicates with another set of computers and application components that are 
housed underneath (the "receiving station") as one component (207) of the apparatus (101). 
[Para 20] A more detail view of the physical structure of the projector camera wall is 
depicted in FIG. 3, wherein there are two components a) projection lens (301) that beams 
the light on the screen (202) and b) digital video camera lens (302) that receives the light 
from the object in the apparatus (101). Both lenses flip to one another, which means a 
single cell has both projection lens and camera lens and when projection lens is active, that 
cell produces light and when it flips to camera lens the lens receives the light. However, 
they do not co-exist. The flipping process (the "flip") between projection lens to camera lens 
at a very high-speed makes this switch seem-less. 

[Para 21] One camera lens is usually active in one track (the "track") where as rest of the 

lenses are on projection mode. This active camera lens on the track moves (the "trace") 

vertically at a very high speed. This trace is guided by a laser light (304) for recording and 

reproducing information. The pattern of trace (the "pattern") is such that a very high speed 

it appears to be moving both horizontally and vertically. Each camera produces a series of 

images from every dimension with each image is being comprised of whole image (the 

whole") as well as part of the whole image (the "part"). The depth information is further 

manipulated by the controller (206) to produce dynamic object-centered descriptions of 

everything within an image. Controller, as a stand-alone system, triggers to synchronously 

record frames from multiple cameras. The output of each camera is time stamped with a 

common Vertical Interval Time Code (the "VITC"). The time code allow us to correlate the 

frames across cameras which is crucial when transcribing movements and triggering effect. 

The combination of using pattern, multi-dimensional whole and part Images from each 

video camera with VITC (the "light receiving process") allow the controller to record in a way 

to form three-dimension dynamic images or holographic images. 
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[Para 22] The projection lens (301), on the other hand, uses the multi-dimensional whole 
and part images on a screen comprising: a coherent illumination means; an electrically 
addressed spatial light modulator (the "EASLM") means configured to diffract light received 
from the coherent illumination means, wherein the light is received by said EASLM means as 
a plurality of sequential computer generated images; and optics configured to direct light 
diffracted by the electrically addressed spatial light modulator means to different areas of 
the screen to optically build up a whole image of the multi dimensional image over time, 
wherein each of the plurality of sequential computer generated images is whole and part of 
the complete image correspond to one of the different areas of the screen (202), thus 
forming the environment on the screen (202) of the apparatus (101). The top and bottom 
section of the screen has sensors (303) to capture movement of the user of the apparatus 
(101). 

[Para 2 3] What follows in FIG. 4, a procedure of communication between the user of the 
apparatus and the IPE environment on a real-time basis. Any physical object environment 
(401) located outside the apparatus, that is present at the time or pre-recorded, is captured 
in identical pattern (404) including audio and sensor output and communicated over the 
wireless communication network (106) to the apparatus (101), which uses projection lens 
(301) to send data to its processor (201) and to the screen (202) to overlay message. If the 
display attribute (403) is for specifying scrolling or moving (synchronized with participant 
movement) the overlay message is displayed in the environment (402) as a scrolled manner 
(giving an impression of movement). If the display attribute (403) is for specifying reverse 
display, flashing , coloring, or display sizing, the overlay message is displayed in the 
environment (402) as specified within the apparatus. 

[Para 24] If the final desired display resolution is N pixels broken up into b blocks. Then the 

controller (207) will process as if each block has N/b pixels and N/b computerized 

projected pixels of each whole picture combined with part of the picture to create the IPE 
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environment or computerized graphics for hologram pattern. Each block is processed in the 
processor (201) and illuminated in sequence to build up a frame, hence each frame is fast in 
order to give the user the whole display as illuminated. If the number of frames produced 
each second is f then the number of EASLM frames per second is given by f*b. This mean's 
that the total number of projected pixels that are combined in each second is (f*b)N. This 
result is independent of the block size and suggests that the only way to bypass the 
computational burden is to run computations in parallel. The field sequential color frame 
rate is around thousand frames per second, and thus for a ten million pixel image produce 
(1 K"l 0)^ projected pixels must be combined per second to create photo-realistic three- 
dimensional dynamic images or holographic images. 

[Para 2 5] The recording of the visual data, audio data and sensor data in the apparatus is 
captured in combination of digital video camera (302) and laser light (304) and audio input 
(104) as depicted in FIG. 3, collectively form a light receiving process (the "light receiving 
process") as depicted in the FIG. 5. They are trigger the laser scanning screen (504), which 
is invisible in nature to measure and record the dynamic object from all dimensions (the 
"multi-dimensional") to form the three-dimensional image. The spatial modulator (501) 
displays a pattern of white and black pixels obtained by executing a two-dimensional digital 
modulation of information. The spatial modulator uses the pattern to modulate the light 
beam. FIG. 5 shows an example of the two-dimensional modulation executed by the spatial 
modulator. In this example, as shown in FIG. 5, digital input information data (502), that is, 
information data in "0" and "1" combinations to be recorded on a recording medium (503), 
is expressed using a combination of white and black pixels. An array of white and black 
pixels arranged In this order In a vertical direction corresponds to the input information 
data "0" combinations. An array of black and white pixels arranged in this order in the 
vertical direction corresponds to the input information data "1" combinations. A laser light 
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(304) guided by two-dimensional sensor (303) detects a detected light from the recording 
medium (503) to reproduce the recording information. 

[Para 26] Next, description of the formation of IPE environment - recording and 
reproduction - in the apparatus according to a first embodiment of the present invention; " 
FIG. 6 is a block diagram schematically showing the configuration of the signal processing 
system of the IPE, which could be generated by, and not limited to, data base, integration of 
data, software application, software code, computer, wireless network hardware, etc 

■ 

recording control of the system is through the data glove or equivalent device, and 
reproduction apparatus according to the first embodiment. The incoming and outgoing data 
to formulate interactivity between the user and IPE environment, as depicted in the FIG 6, 
digitized and processed through a wireless communication network. 
[Para 27] The process of digitization and synchronizing for IPE environment triggers 
processor and controller to coordinate, as depicted in FIG. 7, for interactivity. As the 
interaction with the user and IPE environment changes, the information recorded in the 
computer triggers the change in the environment and updates the environment following 
the same process. Referring to FIG 7, an automated protocol management system that may 
be utilized in the present system of the invention may contain one or more following 
functions: 

[Para 28] the design, and editing of protocols for pre-recorded information (the "stand- 
alone") use and downloading information to an external device 
[Para 29] detail guiding of the user through the process of using the apparatus for 
interactivity 

1 . signaling the start and end of the protocol and all required signals for both 
ends 

2. collecting data and recording them in a recording medium 
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3. uploading data from the "connected" and "stand-alone" apparatus such as 
database and software to provide information to the user 

4. sharing of information between one user to another within the same apparatus 

5. providing a.GPS and tracking methods for user information 

6. providing visual and auditory cues to the users 

The methods, apparatus and systems of the invention have broad uses in a wide range of 
activities and competitive environments. The system not only has applications for using by a 
single community, but also for workplace, shopping mall, retail, banking and other function 
performance. Situations that require methodical monitoring and feedback on speed and 
ability of movement. 
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